6 research outputs found
Towards a Holistic Approach: Understanding Sociodemographic Biases in NLP Models using an Interdisciplinary Lens
The rapid growth in the usage and applications of Natural Language Processing
(NLP) in various sociotechnical solutions has highlighted the need for a
comprehensive understanding of bias and its impact on society. While research
on bias in NLP has expanded, several challenges persist that require attention.
These include the limited focus on sociodemographic biases beyond race and
gender, the narrow scope of analysis predominantly centered on models, and the
technocentric implementation approaches. This paper addresses these challenges
and advocates for a more interdisciplinary approach to understanding bias in
NLP. The work is structured into three facets, each exploring a specific aspect
of bias in NLP
Automated Ableism: An Exploration of Explicit Disability Biases in Sentiment and Toxicity Analysis Models
We analyze sentiment analysis and toxicity detection models to detect the
presence of explicit bias against people with disability (PWD). We employ the
bias identification framework of Perturbation Sensitivity Analysis to examine
conversations related to PWD on social media platforms, specifically Twitter
and Reddit, in order to gain insight into how disability bias is disseminated
in real-world social settings. We then create the \textit{Bias Identification
Test in Sentiment} (BITS) corpus to quantify explicit disability bias in any
sentiment analysis and toxicity detection models. Our study utilizes BITS to
uncover significant biases in four open AIaaS (AI as a Service) sentiment
analysis tools, namely TextBlob, VADER, Google Cloud Natural Language API,
DistilBERT and two toxicity detection models, namely two versions of
Toxic-BERT. Our findings indicate that all of these models exhibit
statistically significant explicit bias against PWD.Comment: TrustNLP at ACL 202
Survey on Sociodemographic Bias in Natural Language Processing
Deep neural networks often learn unintended biases during training, which
might have harmful effects when deployed in real-world settings. This paper
surveys 209 papers on bias in NLP models, most of which address
sociodemographic bias. To better understand the distinction between bias and
real-world harm, we turn to ideas from psychology and behavioral economics to
propose a definition for sociodemographic bias. We identify three main
categories of NLP bias research: types of bias, quantifying bias, and
debiasing. We conclude that current approaches on quantifying bias face
reliability issues, that many of the bias metrics do not relate to real-world
biases, and that current debiasing techniques are superficial and hide bias
rather than removing it. Finally, we provide recommendations for future work.Comment: 23 pages, 1 figur
Nationality Bias in Text Generation
Little attention is placed on analyzing nationality bias in language models,
especially when nationality is highly used as a factor in increasing the
performance of social NLP models. This paper examines how a text generation
model, GPT-2, accentuates pre-existing societal biases about country-based
demonyms. We generate stories using GPT-2 for various nationalities and use
sensitivity analysis to explore how the number of internet users and the
country's economic status impacts the sentiment of the stories. To reduce the
propagation of biases through large language models (LLM), we explore the
debiasing method of adversarial triggering. Our results show that GPT-2
demonstrates significant bias against countries with lower internet users, and
adversarial triggering effectively reduces the same.Comment: Paper accepted in the 17th Conference of the European Chapter of the
Association for Computational Linguistics (EACL2023
Unmasking Nationality Bias: A Study of Human Perception of Nationalities in AI-Generated Articles
We investigate the potential for nationality biases in natural language
processing (NLP) models using human evaluation methods. Biased NLP models can
perpetuate stereotypes and lead to algorithmic discrimination, posing a
significant challenge to the fairness and justice of AI systems. Our study
employs a two-step mixed-methods approach that includes both quantitative and
qualitative analysis to identify and understand the impact of nationality bias
in a text generation model. Through our human-centered quantitative analysis,
we measure the extent of nationality bias in articles generated by AI sources.
We then conduct open-ended interviews with participants, performing qualitative
coding and thematic analysis to understand the implications of these biases on
human readers. Our findings reveal that biased NLP models tend to replicate and
amplify existing societal biases, which can translate to harm if used in a
sociotechnical setting. The qualitative analysis from our interviews offers
insights into the experience readers have when encountering such articles,
highlighting the potential to shift a reader's perception of a country. These
findings emphasize the critical role of public perception in shaping AI's
impact on society and the need to correct biases in AI systems
The Sentiment Problem: A Critical Survey towards Deconstructing Sentiment Analysis
We conduct an inquiry into the sociotechnical aspects of sentiment analysis
(SA) by critically examining 189 peer-reviewed papers on their applications,
models, and datasets. Our investigation stems from the recognition that SA has
become an integral component of diverse sociotechnical systems, exerting
influence on both social and technical users. By delving into sociological and
technological literature on sentiment, we unveil distinct conceptualizations of
this term in domains such as finance, government, and medicine. Our study
exposes a lack of explicit definitions and frameworks for characterizing
sentiment, resulting in potential challenges and biases. To tackle this issue,
we propose an ethics sheet encompassing critical inquiries to guide
practitioners in ensuring equitable utilization of SA. Our findings underscore
the significance of adopting an interdisciplinary approach to defining
sentiment in SA and offer a pragmatic solution for its implementation.Comment: This paper has been accepted and will appear at the EMNLP 2023 Main
Conferenc